JAMIA Open

Cancer registries are essential for population-level surveillance, epidemiologic research, and public health planning, but their accuracy depends on human review of key variables documented in unstructured clinical text. This is time-consuming, and current quality assurance (QA) typically relies on reviewing fewer than 10% of records at random, which can miss facility- or time-specific errors. We developed a large language model (LLM)-based framework for targeted cancer registry QA. The framework uses LLMs to extract biomarker values from clinical text, compares them with registry entries, and flags disagreements for expert review. Six open-source LLMs were benchmarked locally in a privacy-preserving environment, and the best model per site and report type was applied to breast and prostate cancer, two of the most common cancers in the United States. For the four breast cancer biomarkers ER, PR, HER2, and Ki-67, the disagreement rate against quality-assured registry values in 1,852 breast cancer cases was lowest for ER at 1.67% and highest for Ki-67 at 9.18%. In 920 prostate cancer cases, the disagreement rate with the deployed models stayed below 8% for every categorical Gleason variable, while the exact-value PSA metric reached 17.28%. PSA therefore generated the largest volume of candidate discrepancies, though individual PSA flags require more corroboration than categorical flags. Applied to a full year of records (11,740 breast and 6,792 prostate cases), the framework flagged disagreements and assigned each a probable cause, grounded in NAACCR coding rules, to prioritize expert review. Extracted values are linked to their supporting text, and model, prompt, and pipeline versions are logged for reproducibility. By directing review toward records most likely to contain discrepancies, the framework offers an efficient, auditable alternative to random QA sampling for cancer registries.

Metabolic dysfunction-associated steatotic liver disease (MASLD) is highly prevalent among individuals with type 2 diabetes (T2D), yet accurate identification from electronic health records (EHRs) remains challenging because clinically relevant information is fragmented across structured data and unstructured clinical documentation. We developed a rule-based computable phenotype (CP) framework that integrates structured EHRs with clinical notes for scalable MASLD identification in patients with T2D. The framework comprises five complementary CP algorithms derived from validated fibrosis indices and liver injury biomarkers, together with a lightweight natural language processing module that identifies MASLD-related evidence using keyword matching, contextual filtering, and negation detection. The proposed framework was systematically evaluated across multiple threshold settings, with additional validation through age- and sex-stratified analyses and manual chart review. The findings demonstrate that integrating structured and unstructured EHR data provides an accurate and scalable framework for real-world MASLD phenotyping and offers a practical strategy for developing CPs for other chronic diseases.

Background: Large language models (LLMs) have demonstrated strong performance on standardized medical examinations, with recent studies reporting performance approaching or exceeding that of senior medical residents. However, examination accuracy alone does not establish how models arrive at their answers or the relative contributions of the clinical vignette, imaging, and answer options to model performance. We evaluated the contribution of these question components to LLM performance on the Orthopaedic In-Training Examination (OITE). Methods: We evaluated three open-source Ministral-3 models (3B, 8B, and 14B) and five proprietary models (Claude Haiku-4.5, Sonnet-4.6, Opus-4.8, GPT-5.6 Luna, and GPT-5.6 Terra) using 792 OITE questions from 2020 through 2024, comprising 434 questions containing clinical images and 358 without images. Question components were sequentially removed across five image-containing and three non-image-containing conditions. Accuracy was summarized using the median and IQR and compared using paired analyses with Holm adjustment. Semantic similarity between model-generated explanations and reference discussions was assessed using BioMedBERT. Results: For image-containing questions, pooled accuracy was 59.13% (58.21-60.08) with complete information and 59.13% (58.18-60.11) without images. Accuracy decreased to 49.05% (48.07-50.00) without the clinical vignette, 45.68% (44.70-46.69) without the vignette and images, and 36.49% (35.51-37.44) when the vignette, images, and question were removed. For non-image-containing questions, accuracy decreased from 72.94% (72.10-73.85) with complete clinical context to 53.53% (52.41-54.68) without the vignette and 37.36% (36.28-38.48) with answer options alone. OpenAI GPT-5.6 Terra achieved the highest full-context accuracy in both image-containing 81.80% (80.41-82.95) and non-image-containing 94.13% (93.30-94.97) questions. Removal of images alone did not significantly affect accuracy for any model, whereas removal of the clinical vignette significantly reduced accuracy for most models. When provided with only the answer options, without the question stem, clinical context, or images, the observed accuracy for every model exceeded the 25% expected from random selection. Specifically, the highest accuracy was 45.62% (44.24-47.24) for Claude Opus 4.8 on questions originally containing images and 46.09% (44.41-47.77) for GPT-5.6 Terra on questions without images. Although closed-source models achieved substantially higher accuracy than open-source models, all models performed above chance when provided answer options alone. BioMedBERT similarity remained high despite substantial differences in accuracy and was not significantly associated with accuracy by Spearman correlation ({rho}=-0.17; P=0.29). Conclusion: Clinical vignettes were the primary contributor to improving LLM performance on OITE whereas images had minimal effect. The high accuracy when only answer options were given suggests the models may be learning spurious correlations. Furthermore, BioMedBERT semantic similarity remained high across models despite wide differences in accuracy and did not consistently distinguish correct from incorrect responses. Thus, high examination performance may not directly reflect clinical reasoning, highlighting the importance of assessing how LLMs use the information provided when interpreting benchmark performance.

Objective: The goal was to integrate structured and unstructured data from the electronic health record (EHR) to identify social and behavioral determinants of health (SBDH) factors and evaluate their association with high emergency department (ED) utilization among cardiovascular disease (CVD) patients. Methods: A custom natural language processing (NLP) algorithm was developed to extract 17 SBDH domains from clinical notes, combined with ICD-10 Z codes and screening questionnaires from the EHR. The study included patients aged 18-64 with a history of CVD. Multivariable logistic regression was used to identify associations between SBDH and high ED utilization ([≥] 5 visits) after adjusting for age, sex, race, insurance class, and chronic conditions. Results: Use of only ICD-10 codes and screening questionnaires identified patients in only a subset of the 17 SBDH domains, whereas extraction from clinical notes identified patients across all domains and increased prevalence estimates for every domain. Among 4,844 patients with CVD, 526 (10.9%) were frequent ED users. After adjustment, patients with inadequate support system (OR= 2.8), opioid abuse (OR=2.47), inadequate insurance (OR=2.42), alcohol abuse (OR=2.28), medication affordability concerns (OR=2.28), unreliable transportation (OR=1.82), financial strain (OR=1.69), and depression (OR=1.42) were significantly more likely to have frequent ED utilization. Conclusion: NLP can efficiently and more comprehensively identify socioeconomic risk factors from the EHR. Substance use, financial strain, inadequate social support, and other socioeconomic challenges may be linked to increased ED visits in patients with CVD.

Background: Acute kidney injury (AKI) is a severe complication in intensive care units, frequently exacerbated by synergistic nephrotoxicity from drugs such as Vancomycin and Piperacillin-Tazobactam. Traditional alert systems relying on static thresholds suffer from high false-positive rates and delayed detection. Methods: We developed Agentic-TimesFM-AKI, a dual-model architecture integrating a Large Language Model (Gemma-4 Sentinel) with a zero-shot time-series forecaster (TimesFM) to provide continuous, dynamic risk forecasting and transparent clinical reasoning. The system was trained on a synthetically generated cohort with differential privacy ({varepsilon}=10) and evaluated on the publicly accessible eICU (N=200) and MIMIC-IV (N=117) Demo datasets. Results: In the internal eICU pilot evaluation, the framework achieved an Accuracy of 0.970 (95% CI: 0.945-0.990) and an F1-Score of 0.966, successfully mapping temporal physiological trajectories into intelligible natural language alerts. However, external validation on the MIMIC-IV cohort revealed severe performance degradation. Conclusions: While the dual-model framework provides highly accurate and interpretable AKI alerts on familiar schema cohorts, it suffers from structural formatting fragility and domain shift. This highlights critical vulnerabilities in applying generative models to out-of-distribution electronic health records. Keywords: Acute Kidney Injury, Large Language Models, Time-Series Forecasting, Electronic Health Records, Differential Privacy, Pharmacovigilance.

Background Peripheral artery disease (PAD) is a major cause of cardiovascular events but remains underdiagnosed. Electronic health record (EHR)-based machine learning models show promise for earlier detection, but developing generalizable and fair models across diverse populations remains challenging. Methods Using the University of California Health Data Warehouse, containing EHR data from five health systems, we identified patients with and without PAD. We used unsupervised clustering to define PAD phenotypes and trained a LightGBM classifier using 14,023 features spanning demographics, comorbidities, medications, laboratory values, healthcare utilization, and diagnosis, procedure, and medication codes. We evaluated performance overall and across demographic groups and phenotypes, and assessed fairness using selection rates and subgroup differences in true- and false-positive rates. Results The study included 33,739 cases and 33,739 matched controls. Clustering identified four phenotypes: patients with limited healthcare documentation (cluster 1), younger patients with severe metabolic disease (cluster 2), patients with a traditional atherosclerotic risk profile (cluster 3), and frail elderly patients with multimorbidity (cluster 4). Overall, the model demonstrated consistent performance across institutions (AUROC 0.76?0.79; AUC-PR 0.76?0.79) with well-calibrated probabilities. Performance was similar across genders, with modest variation by race and age, and was stronger in clusters 2?4. Cluster 2 demonstrated the highest sensitivity (TPR 0.87, 95% CI 0.87?0.88), while cluster 1 showed the lowest performance (TPR 0.40, 95% CI 0.39?0.41). Conclusions The EHR-based PAD detection model demonstrated consistent performance across five health systems. Phenotypic clustering revealed clinically meaningful differences in model performance adding an additional consideration in ML fairness and performance evaluations.

Objectives Transformer models for electronic health records require converting clinical data into token sequences, however standardized tokenization and evaluation frameworks are lacking. We introduce DBToken, an open-source library, and bits-per-row (BPR), a metric for comparing tokenization strategies. Materials and Methods DBToken accepts Medical Event Data Standard (MEDS)-compatible input and supports multiple text, numeric, and temporal tokenization strategies. BPR extends the bits-per-byte metric used in language models to enable comparison across tokenization strategies. Results DBToken efficiently tokenized data across configurations. BPR identified the vocabulary size associated with the best clinical outcome performance and localized differences in numeric tokenization performance by token class. Discussion Optimal tokenization strategies for medical foundation models are a subject of active research. DBToken enables reproducible tokenization experiments, while BPR efficiently screens vocabulary sizes and numeric representations before downstream evaluation. Conclusion DBToken and the BPR metric provide open-source infrastructure for reproducible EHR tokenization and cross-strategy evaluation.

Introduction: Rare and heterogeneous disease research increasingly relies on privacy-enhancing technologies such as federated learning (FL) to enable cross-institutional collaboration across fragmented datasets. However, data quality assurance in FL can be limited, as individual-level data may not be directly accessible. While schema-dependent inspection offers a partial solution, it requires standardised schemas or resource-intensive frameworks, hindering scalability and collaboration. To address these challenges, we implemented two complementary dashboards and evaluated their interplay. Methodology: We adapted a recognised data quality control framework for re-use of electronic health record (EHR) data using breast cancer records from Centre Leon Berard into two dashboards: (1) an in-house dashboard for intra-clinic quality assessment, and (2) a federated dashboard for inter-clinic quality assessment. Artificial inconsistencies were introduced into distributed datasets mirroring the in-house source to evaluate detection capabilities. Results: The in-house dashboard provided granularity and reliability, pinpointing individual-level inconsistencies, while the federated dashboard enabled cross-institutional pattern detection - trade-offs inherent to their designs. The federated system revealed ecosystem-wide trends inaccessible to single-institution tools, whereas the in-house dashboard provided local validation and thoroughness. Discussion: Our findings confirm a complementary relationship: FL dashboards provide scalable, collaborative oversight but may require additional quality checks, while in-house tools ensure thoroughness at the cost of scalability. A combined model seemingly offers the optimal balance, accommodating both institution-specific needs and collaborative research requirements in evolving, multi-institutional ecosystems.

Stigmatizing language in clinical documentation, which conveys negative stereotypes, attitudes, or judgments toward patients, is a recognized source of documentation bias and is associated with poorer care and adverse health outcomes. Although prior stigma-related research has focused on clinician-written EHR notes, the increasing use of large language model (LLM)-generated documentation in clinical workflows raises new concerns about its potential to reproduce or amplify bias and affect patient safety. In this study, we conducted a large-scale assessment of stigmatizing language in LLM-generated reasoning text on 35 real-world clinical tasks across 107 LLMs. We applied a psychiatrist-validated, natural language processing (NLP) system to detect stigma terms in LLM reasoning text and quantified stigma rates of LLM-generated reasoning texts across 3,745 model-task pairs. Results showed that stigma rates ranged from 0% to 33.33%, with 84.06% of pairs containing stigma terms. Open-source models and reasoning models showed statistically higher stigma rates than proprietary (1.97% vs. 1.60%; p < 0.01) and non-reasoning models (2.35% vs. 1.70%; p < 0.0001), while the stigma rate difference between the general and medical models is not statistically significant (2.00% vs. 1.80%; p = 0.26). Stigma rates of LLM outputs correlated negatively with task accuracy (r = -0.304; p < 0.001) and positively with input clinical-text stigma (r = 0.569; p < 0.001), with 19.76% of model-task pairs amplifying stigma in the original input notes. Applying prompt engineering as a destigmatizing approach helped reduce model stigma rates by as much as 91.91% without affecting the model performance. This study shows that stigmatizing language generation is common but reducible during LLMs' reasoning traces, suggesting that well-implemented approaches for LLM monitoring and destigmatizing will be essential for healthcare systems to implement.

Objective: Retrospective evaluation of clinical pathway adherence using electronic medical record (eMR) data is limited by heterogeneous data structures and statistical measures that do not account for patient-specific context, care sequence, or timing. We designed a modular event-based architecture to dynamically evaluate the New South Wales Adult Sepsis Pathway over time. Method: Data from eMRs were collected from adult emergency department encounters across four hospitals between July 2022 and June 2024 and processed through three sequential layers: event-stream transformation, pathway-relevance filtering, and state-based pathway evaluation. Heterogeneous records were standardized and organized into a chronological timeline for each encounter. The framework then updated the pathway state as new clinical information became available and assessed whether recommended care was delivered within the relevant timeframe and sequence. Result: The architecture processed 451,124 encounters. This included 57,631,084 clinical events extracted and transformed into encounter-level event streams. After filtering for pathway relevance, 43,951,146 events remained (76.3%). A total of 129,907 (34.6%) encounters entered the sepsis pathway, 50,118 (38.6%) entered YELLOW state, indicating progression to suspected sepsis, and 3,349 (2.6%) encounters progressed to RED state from GREEN, indicating the initial shock. Of 50,118 encounters entering YELLOW state, 2,426 (4.8%) progressed to shock, 8,700 (17.4%) encounters were recognized with sepsis through a blood culture order, and 4,287 (8.6%) encounters were recognized with sepsis through an antibiotic order. This demonstrates the ability of the architecture to prioritize higher-acuity states during evaluation rather than going through an intermediate state. Conclusion: The architecture developed shows the feasibility of transforming fragmented routine eMR data into a reusable longitudinal representation that supports context-dependent pathway evaluation. Separating data preparation from pathway rules improves transparency, maintainability, and adaptability and provides a foundation for future interoperable, near-real-time pathway monitoring

Background. Research data enclaves such as the All of Us Research Program do not allow participant-level data, or models fitted to them, to leave, which makes it hard to deliver case-finding models, which flag patients with possibly uncoded conditions for clinician review, to patients and clinicians. Objectives. We describe and technically evaluate a system that fits models outside an enclave from released summary counts and delivers them through standard health information technology interfaces. Methods. We exported dissemination-compliant summary counts from the All of Us Researcher Workbench and, outside the enclave, fitted versioned models for Long COVID, chronic fatigue, fibromyalgia, endometriosis, Lyme disease, lupus, migraine, depression, and anxiety. A model contract (a model file and a written specification), a Python reference implementation, and synthetic Fast Healthcare Interoperability Resources (FHIR) test patients define the expected outputs. One JavaScript scorer runs in a patient demonstration, a SMART on FHIR clinician application, and a CDS Hooks service. An atlas displays aggregate results. We evaluated the system on synthetic records with automated tests, cross-implementation output comparison, launches through the public SMART sandbox, and a counterexample review of the scoring and display rules. Results. The deployed Long COVID model had an area under the receiver operating characteristic curve of 0.67 in a random validation split. The eight self-report models reached 0.61 to 0.82 in identifying self-reporters without a code. The JavaScript scorer reproduced all 162 reference outputs exactly, and all 437 automated tests passed. A counterexample review found 5 classes of failure, including scoring ineligible patients and treating unavailable data as absent, all fixed and added as tests. Conclusions. Models fitted from released summary counts can be delivered through FHIR, SMART on FHIR, and CDS Hooks with reproducible scoring on synthetic records. Real-record interoperability, usability, and prospective clinical performance remain to be evaluated.

Background: Clinical research often requires reviewing large volumes of unstructured electronic medical record (EMR) data, a time-consuming task demanding skilled personnel. Large language models (LLMs) like ChatGPT can efficiently analyze and summarize text, potentially accelerating chart review research. However, concerns about personal health information (PHI) leakage limit use of commercial chatbots. Secure, in-house LLMs within fire-walled environments can address these concerns. Aim: To develop a user-friendly, scalable application leveraging an in-house ChatGPT 4o-mini on Emplify Health's Azure cloud to automate processing of tabular clinical data while protecting PHI. Methods: The application imports tabular data (Excel or text), guides users through prompt engineering, and automatically submits data row-by-row to the LLM. It retrieves and tabulates results as specified by users, enabling both automated extraction of clinical parameters, and data summarization/interpretation. Results/Conclusions: Initially designed to identify cancer cases and extract related parameters from pathology reports, a fully generalized application was developed which has found utility in analyzing diverse data sources including cancer registries, cardiology CT reports, imaging narratives, and clinical notes. This tool has significantly accelerated research by reducing data retrieval time, allowing staff to focus on higher-value tasks like data analysis, while maintaining PHI protection.

Objective: Transformer-based models for clinical prediction using longitudinal electronic health record (EHR) data are often developed with large feature sets and long patient histories under the assumption that more data improves performance. However, high-dimensional inputs and long sequences increase computational burden, potentially limiting scalability. We evaluated how feature dimensionality and sequence length affect predictive performance, calibration, risk stratification, and computational efficiency in EHR prediction. Methods: Using longitudinal EHR data from adults with mild asthma in an integrated healthcare system, we evaluated input representation design for predicting acute asthma exacerbation. Feature dimensionality was reduced using Integrated Gradients attribution scores, univariate performance-based selection, and clinically guided selection strategies. Sequence length was varied using percentile-based truncation of patient histories. Performance was assessed using discrimination, calibration, high-risk classification, threshold-based event capture, and computational efficiency across regions. Results: Models using fewer features achieved discrimination comparable to the 80-feature reference model, with AUROC values ranging from 0.843 to 0.864 versus 0.870 for the full model. Moderate sequence-length truncation reduced training time by more than 70% with minimal loss in discrimination. Although reduced-dimensional models showed attenuation of predicted risk at the upper tail, they identified similar high-risk populations and captured comparable proportions of asthma exacerbation events at clinically relevant thresholds. Conclusion: Transformer-based prediction models maintained strong performance across reduced feature sets. While dimensionality reduction modestly affected calibration at the highest risk levels, moderate sequence-length reduction substantially reduced computational burden with limited change in overall discrimination. These findings highlight trade-offs between input complexity, predictive performance, and computational efficiency.

As the use of electronic health records in drug repositioning research increases, so does the need for a well-curated resource describing real-world drug-diagnosis relationships. This need is particularly important in the context of polypharmacy. Although literature-based drug-disease maps exist, they are typically based on mechanistic disease ontologies, which are not widely used in clinical settings and do not align well with the ICD system, which is most often used in healthcare. Here, we used real-world primary and secondary healthcare data from approximately 1.5 million individuals to identify drug-diagnosis co-occurrences (~736,000 pairs), significant associations (7,763 pairs), and assess direct drug usage through medical expert curation. The final resource comprises 7,763 associations with odds ratios [&ge;]3.5, manually annotated by six independent clinicians (3 in the UK and 3 in Denmark). Finally, the clinician annotations were scored using an Expectation-Maximization-based framework providing a confidence score for each pair. Our resource shows that the vast majority of significantly associated drugs and diagnoses in healthcare records are not due to direct treatment of the diagnosis. Additionally, through the annotation results, we demonstrate the importance of accounting for systematic differences among annotators when working with real-world data.

Objective: We explored the implementation considerations of machine learning (ML) suicide risk identification in a health system that serves primarily American Indian patients in the Southwestern United States. Previous work developed and reported the performance of suicide risk models, but traditional performance metrics do not directly reflect clinical workflows. Materials and Methods: Using EHR data collected for Emergency Department (ED) visits between 10/01/2019 and 10/02/2021, we conducted Decision Curve Analysis (DCA) to identify the standardized net benefit (SNB) of all models previously evaluated and plotted the potential workload implications of the model with the greatest SNB. Finally, we evaluated suicide screening practices during the study period to inform implementation barriers. Results: Among 9,244 patients, 139 (1.5%) had a suicide attempt or death within 90 days of an included visit. The population-specific logistic regression model had the greatest SNB compared to other approaches. When operationalized, the 95th percentile cutoff did not substantially increase clinical workloads, while the 75th percentile cutoff substantially impacted workloads. The secondary analysis on suicide screening depicted variations in screening administration over time. Discussion: While DCA identified advantages of ML suicide risk identification over current standards, the complexities of suicide-related care limited its clinical interpretability. A stepped-care framework may mitigate potential workload burdens. Training initiatives and dynamic implementation strategies will support high-quality implementation of ML suicide risk identification. Conclusion: Our analyses support the feasibility and utility of augmenting existing suicide screening practices with ML suicide risk identification, and demonstrate the importance of nuanced analyses to inform implementation considerations.

In this work, we demonstrate the unprecedented value of NIH's "All of Us Research Program" (AoURP) dataset in studying maternal morbidity and building predictive machine learning (ML) models across heterogeneous populations in the United States. We developed robust and data-driven preprocessing pipelines to curate a longitudinal, multi-site, multimodal, and demographically diverse pregnancy dataset (20,253 subjects; 27,525 pregnancy episodes) from AoURP data, using electronic health records (EHR) (Conditions, Labs, Measurements) and survey responses (Social Determinant of Health (SDoH)), focusing on 7 crucial maternal health adverse outcomes. After characterizing data quality, missingness, and heterogeneity, we performed statistical correlation analysis to identify risk factors. We subsequently developed XGBoost and sequential LSTM models to predict the adverse outcomes, reaching state-of-the-art performance for multiple outcomes. We conducted model interpretability post-hoc analysis to understand success points and fairness analysis to evaluate implications for socio-economic disparities. Four practicing physicians reviewed the set of statistically significant and ML model identified features to assess their clinical validity and novelty. Most features identified through either statistical correlations or ML feature importance analysis aligned with known clinical risk factors. Several features were identified that the ML models used but that are not currently used in clinical practice and may merit further clinical investigation. Fairness analysis revealed certain associations with SDoH and age highlight areas that warrant continued monitoring. Overall, we demonstrate that meaningful populational level patterns can be extracted, and high-performing machine learning models can be trained on this longitudinal, diverse, multi-site dataset. Important risk features, particularly novel ones identified, if validated, could inform new strategies for maternal care or enable development and validation of outcome-specific, clinically deployable ML models.

Objective: To develop and evaluate machine learning methods for automated identification of family relationships from electronic health records (EHRs). We employed Random Forest classifiers to identify five relationship types: Mother-Child, Father-Child, Sibling-Sibling, Twin-Twin, and Partner-Partner, using a four-stage iterative refinement process to improve precision for patient-centered outcomes research (PCOR) and healthcare applications. Materials and Methods: We used two large-scale Indiana datasets: the Indiana Network for Patient Care (INPC), comprising approximately 15 million unique individuals across approximately 45 million medical records, and the Indiana Natality dataset (birth certificates, 1970-2024), which served as the gold standard. Positive cases were derived by linking verified relationships from Natality records to INPC using a shared Global Identifier. Negative cases were drawn from a three-tier blocking strategy combined with sliding window restriction and similarity scoring, which together reduced the comparison space from over 1014 potential pairs to approximately 121 million candidates. Five Random Forest classifiers underwent four- stage iterative refinement: initial training, feature removal for multicollinearity, feature engineering to address systematic errors, and training-data refinement restricted to relationship pairs with at least one shared contact feature. Results: The models achieved precision of 0.92 to 1.00, recall of 0.97 to 1.00, and F1 scores of 0.94 to 1.00 across all relationship types (Mother-Child 0.97, Father-Child 0.98, Sibling-Sibling 0.98, Twin-Twin 1.00, Partner-Partner 0.94). Between 15% and 78% of Natality-verified relationships lacked any shared contact information in INPC and were structurally undetectable by record linkage; restricting training and evaluation to linkable pairs improved precision, and training-data refinement improved F1-scores by a further 0.02-0.04 for the Mother-Child, Father-Child, and Sibling models. High-confidence predictions (probability [&ge;] 0.9) captured 77-99% of true positives. Age difference was the primary predictor for parent- child and twin relationships, while phone number similarity was most important for siblings. Discussion: The framework identifies family relationships from standard demographic fields available in most EHR systems. The iterative refinement approach shows how systematic error analysis can guide 1 methodological improvements. Training-data refinement addresses a constraint in EHR-based family linkage: not all verified biological relationships have overlapping demographic footprints in healthcare data. Unlike rule-based approaches, the framework provides probability scores that enable configurable deployment thresholds. Conclusion: The Random Forest models demonstrate high performance suitable for large-scale research deployment and are ready for testing in clinical applications rather than immediate widespread clinical use. The methodology can be adapted to other EHR systems, supporting family-centered study design in patient- centered outcomes research. Keywords: family linkage, electronic health records, machine learning, Random Forest, record linkage, patient-centered outcomes research, health informatics

Background Polyendocrine Metabolic Ovarian Syndrome (PMOS), formerly known as Polycystic Ovary Syndrome (PCOS), is a prevalent endocrine disorder with high rates of undiagnosed cases globally. Accessible screening tools are needed to facilitate appropriate management and earlier intervention. As PMOS is frequently characterised by oligo-anovulation, the absence of the characteristic rise in basal body temperature typically seen in ovulatory cycles may serve as a physiological marker for the condition. Objective This study aimed to assess the feasibility of using machine learning to identify individuals with PMOS from temperature data collected by a body-worn device. Methods We used data from 387 users of a vaginal temperature monitor (OvuSenseTM) who responded to a questionnaire. The sample was restricted to individuals with at least three cycles with sufficient temperature data and whose PMOS case/control status could be determined from questions about prior clinical consultation for infertility and conditions for which they take medications. We randomly sampled three menstrual cycles for each participant and derived a set of cycle-level and user-level temperature features. Cycle-level features included cycle length and measures describing the temperature rise indicative of ovulation (e.g. temperature rise, cycle day of temperature rise start). We also constructed a reference cycle representing the typical bi-phasic cycle pattern (created using cycles from those without known fertility conditions) and used this to derive features describing how much a participant's cycles differed from this reference. The cycle-level features were aggregated into user-level features by taking the minimum, maximum, median, and range of the cycle-level features across the three selected cycles for each participant. We used 5-fold nested cross-validation to evaluate the extent that PMOS could be predicted, at the cycle and user levels, using Logistic Regression (LR), Support Vector Machine (SVM), and Random Forest (RF). Results The average age of participants was 31.97 years (SD=4.58), with 49.6% having a self-reported PMOS diagnosis. The models demonstrated moderate discrimination, with cycle-level AUC-ROC scores ranging from 0.64 (SD=0.02) (LR) to 0.68 (SD=0.04) (RF), and user-level scores ranging from 0.65 (SD=0.07) (LR) to 0.70 (SD=0.04) (RF). All models were reasonably calibrated, though confidence intervals were wide (e.g. RF cycle-level: calibration slope = 0.83 (95% confidence interval [CI]: 0.68, 1.00), calibration intercept = 0.02 (95% CI: -0.11, 0.14); user-level: slope = 0.88 (95% CI: 0.69, 1.15), intercept = -0.01 (95% CI: -0.22, 0.16)). Conclusions This study demonstrates the potential of using body temperature from digital health devices to identify those with PMOS. Such a passive approach to identifying PMOS could help to identify undiagnosed PMOS in those who have not actively sought a diagnosis. Further research is needed to assess its predictive performance and acceptability in a general population using more widely used digital devices.

Background. Health systems answer most questions by having expert analysts hand-write queries against a complex electronic health record data warehouse, a slow, resource-intensive process. Whether an autonomous coding agent can do this accurately is unknown. Methods. In a single-center quality-improvement evaluation, we posed ten questions about a common pediatric infection, acute otitis media. The questions went to analysts, whose adjudicated answers were the reference, and to an autonomous coding agent (OpenAI Codex), which wrote and ran read-only queries on a full copy of the production data warehouse (Epic Caboodle). We ran the agent under four conditions: an autonomous baseline (each question answered three times at two reasoning-effort settings), a variant in which it listed its assumptions, interactive analyst feedback, and reuse of a corrected definition across related questions. Outcomes were accuracy, patient-level agreement (F1), reproducibility, and cost. Results. Working autonomously, the agent wrote valid queries and never fabricated data, but rarely produced the exact answer. At medium effort, it came within 5% of the reference on 27 of 30 runs but exact on only 10. Higher effort produced no improvement. Reproducibility was the greater weakness, with the three runs returning an identical answer on only 3 of 10 questions. When a run matched the reference, it had found the same patients (F1 = 1.00); the exception was an over-counted procedure (F1 = 0.72). With analyst feedback on four questions, it answered two exactly, and reusing the corrected definition on related questions restored reproducibility and accuracy. Conclusions. A general-purpose coding agent matched an adjudicated analyst reference on most routine questions but not reproducibly, because key definitions depended on warehouse knowledge that the data dictionary omits. Supplying that knowledge, by prompt or feedback, restored reproducibility. Under expert analyst supervision, the agent is already a capable drafting aid and a promising step toward broader hospital analytic support.

Diagnostic errors, including misdiagnoses and delayed clinical diagnoses, could affect outcomes of a significant patient population, particularly individuals presenting with rare diseases or non-specific symptoms. From rule-based diagnostic decision supporting systems (DDSS) to large language model (LLM) based tools for clinical reasoning have been developed to address these limitations. However, existing DDSS are often proprietary and difficult to integrate, and recent LLM-based tools remain hindered by operational challenges such as cost, resources constraint, and privacy concerns. Moreover, existing systems interpret electronic medical records (EMR) and generate diagnoses separately, limiting continuous evidence-based analysis and imposing repeated clinician involvement. In this paper, we present DDx-Finder, an open-source framework that leverages Model Context Protocol (MCP) servers for direct EMR and literature access, enabling prompt-driven clinical state extraction and reliable case-report re- trieval via generating searching query by LLM, while addressing limitations related to resource demands and privacy concerns. A clinical case study demonstrates the systems feasibility and its potential to provide accessible, transparent, and systematic differential diagnostic support for complex cases.